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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments witli respect to claims 21 , 44, and 49 have been 
considered but are moot in view of the new ground(s) of rejection. Furthermore, the 
new reference (Zimmerman) also provides further support for claim 23 as described 
below in the rejection of claim 23. 

2. Applicant's arguments filed November 9, 201 0 have been fully considered but 
they are not persuasive. 

The Applicant argues that Chambers et al. in view of Suzuki et al. fail to 
disclose the limitations "automatically inserting at least one cue with respect to 
the audio based at least in part on a signal received from an automated lighting 
system used to light a live performance" in claim 24, "wherein the signal from the 
automated lighting system is a spotlight-on signal, a spotlight color signal, or a 
spotlight position signal" in claim 25, or "inserting at least one cue with respect to 
the audio based at least in part on monitoring of stage light effects" in claim 26". 
The Examiner respectfully disagrees. Suzuki et al. discloses in Fig. 9, section 81 
a light area that can be seen to have cues in the row marked lighting. 
Furthermore, Suzuki et al. discloses in col. 2, lines 35-43 and col. 14, lines 37-53 
acquiring lighting information along with all of the background information going 
on during the performance. It is well-known in the art that lights during a 
performance would include automated lighting, spotlight-on lighting, and stage 
light effects. Suzuki et al. also discloses in col. 12, line 30 - col. 15, line 38 a cue 
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insertion interface allowing the user to designate points in the presentation they 
would like to be cued. Therefore, Suzuki et al. meets the claimed limitations and 
the rejection is maintained. 

Regarding the arguments for claims 29-31, a reference (Larson) is being 
supplied for the Official Notice that was originally taken for this claim. 

The Applicant argues that Chambers et al. in view of Suzuki et al. fail to 
disclose the limitations "using a mixing board automation to generate at least one 
cue" in claim 32, or "using a track pan value to generate a cue" in claim 33. The 
Examiner respectfully disagrees. As been stated above, Chambers discloses in 
paragraph [0005] the use of a mixing board and the track pan value can be 
determined from looking at the distribution of the left and right channels as seen 
in Figs. 3-5. Suzuki et al. discloses in col. 12, line 30 - col. 15, line 38 a cue 
insertion interface allowing the user to designate points in the presentation they 
would like to be cued along with analyzing the volume of the audio (col. 1 8, lines 
14-26). Furthermore, Suzuki et al. has already been combined with Chambers et 
al. at this point and therefore the use of a mixing board and track pan values by 
Chamber et al. can be used in the aid of inserting a cue into the presentation 
data. Therefore, Chambers in view of Suzuki et al. meet the claimed limitations 
and the rejection is maintained. 
3. In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 41 3, 208 
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USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 21 -28, 32-36, 38-41 , and 43-53 are rejected under 35 U.S.C. 1 03(a) as 
being unpatentable over Chambers et al. (U.S. Patent Application Publication 
2003/0236581) in view of Suzuki et al. (U.S. Patent 6,245,982) in view of Zimmerman 
(U.S. Patent 6,41 1 ,289). 

Regarding claim 21, Chambers et al. discloses a method of providing 
synchronization of a video presentation with an audio presentation, comprising: 
providing for display on a user system an interactive user interface (Figs. 3-5), 
the interactive user interface including: an audio waveform corresponding to 
digital samples of audio over time (Fig. 3); time information displayed in 
association with the audio waveform (time information can be seen on the x-axis 
of the audio waveform diagram in the window (104) in Fig. 3). However, 
Chambers et al. fails to disclose a cue insertion interface that enables a user to 
insert a cue at one or more locations with respect to the audio waveform, wherein 
the cue is configured to cause a modification with respect to the abstract visual 
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presentation in synchronization with the audio presentation when the audio 
presentation is audibly played back, with the abstract visual presentation, via a 
playback device associated with a viewer of the abstract visual presentation, 
wherein the viewer playback device is separate from the editing system; 
receiving a first signal from a user input device to designate a cue at a first 
location with respect to the audio waveform; and storing the designated cue in 
computer readable memory. 

Referring to the Suzuki et al. reference, Suzuki et al. discloses a method 
of providing synchronization of a video presentation with an audio presentation, 
comprising: a method of providing synchronization of a video presentation with 
an audio presentation (Figs. 6, 8, and 9), comprising: a cue insertion interface 
that enables a user to insert cue at one or more locations with respect to the 
audio waveform (Fig. 6 - two types of cues can be seen being used to designate 
information, the arrows and the bars; col. 12, line 30 - col. 13, line 3 - the 
operator can edit the cues; col. 13, line 28 - col. 15, line 38 - motion and scene 
components (cues) can be edited according to the user's liking), wherein the cue 
is configured to cause a modification with respect to the abstract visual 
presentation in synchronization with the audio presentation when played back 
(col. 18, lines 14-26 - the motion waveform is changed according to the motion 
components); receiving a first signal from a user input device to designate a cue 
at a first location with respect to the audio waveform (Fig. 6 - two types of cues 
can be seen being used to designate information, the arrows and the bars; col. 
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12, line 30 - col. 13, line 3 - the operator can edit the cues; col. 13, line 28 - col. 
1 5, line 38 - motion and scene components (cues) can be edited according to the 
user's liking); and storing the designated cue in computer readable memory (col. 
14, lines 8-13). 

Therefore, It would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have allowed the user to edit and insert cues 
along the audio waveform as disclosed by Suzuki et al. in the method disclosed 
by Chambers et al. in order to modify the visual presentation of the audio during 
playback to further engage the user. However, Chambers et al. in view of Suzuki 
et al. still fail to disclose the cue is configured to cause a modification with 
respect to the abstract visual presentation in synchronization with the audio 
presentation when the audio presentation is audibly played back, with the 
abstract visual presentation, via a playback device associated with a viewer of 
the abstract visual presentation, wherein the viewer playback device is separate 
from the editing system. 

Referring to the Zimmerman reference, Zimmerman discloses a method of 
providing synchronization of a video presentation with an audio presentation, 
wherein the cue (the cue is the scaled numeric table which is created in Fig. 3) is 
configured to cause a modification with respect to the abstract visual 
presentation in synchronization with the audio presentation when the audio 
presentation is audibly played back, with the abstract visual presentation (Fig 2 is 
the abstract visual presentation of the audio presentation; col. 4, lines 2-4; col. 2, 
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lines 43-52), via a playback device associated with a viewer of the abstract visual 
presentation, wherein the viewer playback device is separate from the editing 
system (Fig. 1 depicts the apparatus the abstract visual presentation is 
reproduced on; col. 4, lines 7-14 - the memory stores all of the cue information 
(tables) which can be created elsewhere and stored on a portable storage 
medium). 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have had a separate device for the visual 
presentation where the cues are used to modify the visual presentation as 
disclosed by Zimmerman in the method disclosed by Chambers et al. in view of 
Suzuki et al. in order to provide any user with a graphical presentation of the 
audio data. 

Regarding claim 22, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising providing for display via the 
interactive user interface at least left and right audio channel waveforms 
(Chambers et al.: Figs. 3-5 - reference characters (106) and (108) along with 
(104) display the left and right channels; paragraph [0097]). 

Regarding claim 23, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 , but fails to explicitly disclose that the first signal indicates the beginning 
of a guitar rift. However, Suzuki et al. discloses in Fig. 9 capturing the waveform 
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of the guitarist (col. 14, lines 25-36). Suzuki et al. also discloses a cue insertion 
interface that enables a user to insert cue at one or more locations with respect 
to the audio waveform (Fig. 6 - two types of cues can be seen being used to 
designate information, the arrows and the bars; col. 12, line 30 - col. 13, line 3 - 
the operator can edit the cues; col. 13, line 28 - col. 15, line 38 - motion and 
scene components (cues) can be edited according to the user's liking). 
Furthermore, Zimmerman discloses in Fig. 4D a list of instruments that can be 
represented in the visual presentation (col. 9, lines 9-26). Therefore, the user 
would be able to insert a cue to mark the guitar if one desired. 

Regarding claim 24, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising automatically inserting at 
least one cue with respect to the audio based at least in part on a signal received 
from an automated lighting system used to light a live performance (Suzuki et al.: 
Fig. 9 - section 81 ; col. 2, lines 35-43; col. 14, lines 37-53). 

Regarding claim 25, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claims 21 and 24 including that the signal from the automated lighting system is a 
spotlight-on signal, a spotlight color signal, or a spotlight position signal (Suzuki 
et al: Fig. 9 - section 81 - this signal signifies when the spotlight was on; col. 2, 
lines 35-43; col. 14, lines 37-53). 
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Regarding claim 26, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising inserting at least one cue 
with respect to the audio based at least in part on monitoring of stage lighting 
effects (Suzuki et al: Fig. 9 - section 81 - this signal signifies when the spotlight 
was on; col. 2, lines 35-43; col. 14, lines 37-53). 

Regarding claim 27, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising inserting at least one cue 
with respect to the audio based at least in part on a singer's singing (Suzuki et 
al.: Fig. 9 - section (81) - the singers are monitored - Chorus (P4); Fig. 6 - two 
types of cues can be seen being used to designate information, the arrows and 
the bars; col. 12, line 30 - col. 13, line 3 - the operator can edit the cues; col. 13, 
line 28 - col. 15, line 38 - motion and scene components (cues) can be edited 
according to the user's liking). 

Regarding claim 28, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising inserting at least one cue 
with respect to the audio based at least in part on information from a microphone 
and/or based at least in part on information from a vibration sensor located on or 
near an instrument (Suzuki et al.: Figs. 4A-4C, 7A-7E, and 8; col. 8, line 57 - col. 
9, line 61). 
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Regarding claim 32, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising using mixing board 
automation to generate at least one cue (Chambers et al.: paragraph [0005]). 

Regarding claim 33, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising using a track pan value to 
generate a cue (Chambers et al.: the track pan value can be determined from 
looking at the distribution of the left and right channels as seen in Figs. 3-5). 

Regarding claim 34, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the method further comprising using track fader 
adjustments, bus volume, and/or effects send and return levels to generate one 
or more cues (Suzuki et al.: col. 18, lines 14-26 - volume). 

Regarding claim 35, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21, including using an output from a reverb device and/or compressor 
device to generate one or more cues (Suzuki et al.: col. 14, lines 25-36 - motion 
waveform for a guitar; when hooking up the components on the guitar to acquire 
the motion waveforms one could attach a sensor on to the reverb unit). 

Regarding claim 36, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
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claim 21 , but fails to disclose that the method further comprises providing for 
display text describing the cue with the cue, and providing for display abbreviated 
text describing a second cue in association with the second cue, wherein the 
abbreviation is performed at partly in response to a spacing of the second cue 
with respect to another cue. Official Notice is taken that It Is well-known In the art 
to add text to a display to explain what is going on. Therefore, It would have 
been obvious to one of ordinary skill in the art at the time the invention was made 
to have provided for display text describing the cue with the cue, and providing 
for display abbreviated text describing a second cue In association with the 
second cue, wherein the abbreviation is performed at partly In response to a 
spacing of the second cue with respect to another cue in the method disclosed 
by Chambers et al. in view of Suzuki et al. in order to notify the user of what is 
going on during the audio presentation, which will further enhance the user's 
experience. 

Regarding claim 38, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 Including that the cue is a mood cue (Suzuki et al.: col. 15, lines 17-30- 
effects-applylng processes - all of these effect applying processes will create a 
certain mood for the viewer when the viewer plays back the presentation). 

Regarding claim 39, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 including that the designated cue indicates the location of a beat in the 
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audio waveform (Suzuki et al.: col. 17, lines 50-67 - beats are part of tine tempo 
of the music). 

Regarding claim 40, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 Including that the cue Includes a cue identifier Indicating a cue type and 
data Indicating a visualization engine that the cue identifier follows (Suzuki et al.: 
Fig. 6 - two types of cues can be seen being used to designate information, the 
arrows and the bars; col. 1 2, line 30 - col. 1 3, line 3 - the operator can edit the 
cues; col. 13, line 28 - col. 15, line 38 - motion and scene components (cues) can 
be edited according to the user's liking). 

Regarding claim 41, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 Including that the designated cue Is Included In a file separate from the 
audio presentation (Suzuki et al.: col. 17, lines 39-49 - each motion component 
is stored in the motion database, which is separate from the music file). 
However, Chambers et al. in view of Suzuki et al. fails to disclose that the 
method further comprises accessing the file over a network separately from the 
audio. Official Notice Is taken that it is well known In the art to access files over a 
network. Therefore, it would have been obvious to one of ordinary skill In the art 
at the time the invention was made to have accessed the file containing the cues 
over a network in the method disclosed by Chambers et al. in view of Suzuki et 
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al. in order to allow the method to be more versatile and user friendly by allowing 
the user to have access to more files that can be found over a network. 

Regarding claim 43, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 Including that the method further comprises: accessing the designated 
cue from memory; accessing the digital audio samples from memory; providing 
the audio presentation for display in association with the visual presentation 
using the designated cue (Suzuki et al.: Fig. 6 - two types of cues can be seen 
being used to designate information, the arrows and the bars; col. 12, line 30 - 
col. 1 3, line 3 - the operator can edit the cues; col. 1 3, line 28 - col. 1 5, line 38 - 
motion and scene components (cues) can be edited according to the user's 
liking; col. 17, lines 39-49 - playback; col. 18, lines 14-26 - the motion waveform 
is changed according to the motion components)., 

Regarding claims 44-48, these are medium claims corresponding to the 
method claims 21 , 22, 24, 40, and 41 respectively. Therefore, claims 44-48 are 
analyzed and rejected as previously discussed with respect to claims 21 , 22, 24, 
40, and 41. 

Regarding claims 49-53, these are apparatus claims corresponding to the 
method claims 21 , 22, 24, 40, and 41 respectively. Therefore, claims 49-53 are 
analyzed and rejected as previously discussed with respect to claims 21 , 22, 24, 
40, and 41. 
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6. Claims 29-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chambers et al. in view of Suzuki et al. in view of Zimmerman as applied to claim 21 
above, and further in view of Larson (U.S. Patent 5,440,756). 

Regarding claims 29, 30, and 31, Chambers et al. in view of Suzuki et al. 
in view of Zimmerman discloses all the limitations as previously discussed with 
respect to claim 21 , but fails to disclose analyzing a song by inserting at least 
one cue with respect to the audio based at least in part on a filter analysis on the 
power of a plurality of audio frequency bands, wherein the filter analysis cue 
includes a value to indicate an audio frequency band's strength over an interval 
of time, or wherein the filter analysis cue includes an indication that a signal of a 
selected frequency component of having a strength above a predetermined 
threshold value is present in the audio waveform. 

Referring to the Larson reference, Larson discloses performing audio 
analysis using a filter to determine the frequency components of the audio (col. 
72, lines 17-25 and 32-33). 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have included a filter to analyze the audio 
according to the frequency components as disclosed by Larson in the method 
disclosed by Chambers et al. in view of Suzuki et al. in view of Zimmerman in 
order to further enhance the display to match the audio more precisely. 
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7. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chambers in view of Suzuki et al. Zimmerman as applied to claim 1 above, and further 
in view of Kryuchkov et al. (U.S. Patent Application Publication 2004/0102244). 

Regarding claim 37, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 21 , but fails to disclose that the designated cue is a rotation cue indicating 
a rotation speed of at least a first displayed object. 

Referring to the Kryuchkov et al. reference, Kryuchkov et al. discloses a 
presentation device wherein rotation cue (rotation information) indicating a 
rotation speed of at least a first displayed object as inputted by the user 
(paragraph [0205]). 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have allowed the user to input information in 
order to determine the rotation of an object while being displayed in the method 
disclosed by Chambers et al. in view of Suzuki et al. in view of Zimmerman in 
order to enhance the user's experience. Furthermore, the information that is 
being inputted into the Kryuchkov et al. reference regarding the rotation of an 
object can be translated into inserting a cue into the Suzuki et al. reference in 
order to control the presentation display. 

8. Claim 42 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chambers et al. in view of Suzuki et al. in view of Zimmerman as applied to claim 21 , 
and further in view of Nishitani et al. (U.S. Patent 7,161,079). 
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Regarding claim 42, Chambers et al. in view of Suzuki et al. in view of 
Zimmerman discloses all the limitations as previously discussed with respect to 
claim 1 , but fails to disclose that the designated cue is included embedded with 
the audio presentation. 

Referring to the Nishitani et al. reference, Nishitani et al. discloses a 
method wherein the designated cue is included embedded with the audio 
presentation (Fig. 6; col. 8, lines 29-37). 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have embedded the cues with the audio 
presentation as disclosed by Nishitani et al. instead of in a separate file as 
disclosed by Suzuki et al. in the method disclosed by Chambers et al. in view of 
Suzuki et al. in view of Zimmerman in order to easily correlate the audio sample 
with the cue rather than having to read two separate files and trying to correlate 
them. Also, embedding the cues into the audio presentation allows the cues to 
always be accessible because if they were stored in a separate location the other 
location may be unavailable for some reason. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to HEATHER R. JONES whose telephone number is 
(571)272-7368. The examiner can normally be reached on Mon. - Thurs.: 7:00 am - 
4:30 pm, and every other Fri.: 7:00 am - 3:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Peter-Anthony Pappas can be reached on 571 -272-7646. The fax phone 
number for the organization where this application or proceeding is assigned is 571 - 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to ttie Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Heather R Jones 
Examiner 
Art Unit 2481 
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January 15, 2011 

/Peter-Anthony Pappas/ 

Supervisory Patent Examiner, Art Unit 2481 



